Skip to content

fix(tests/cpu-monitor): use psutil.Process for threads to fix errors #5385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 15, 2025
Merged

fix(tests/cpu-monitor): use psutil.Process for threads to fix errors #5385

merged 1 commit into from
Aug 15, 2025

Conversation

Manciukic
Copy link
Contributor

@Manciukic Manciukic commented Aug 14, 2025 •
edited
Loading

Changes

Use a single psutil.Process to verify the process still exists, and extend it with better support for threads using psutil.Process to retrieve their information.

Reason

The previous fix didn't work because the read can also return ProcessLookupError. There is also the issue of pid recycling.

Testing

Example:

integration_tests/performance/test_block.py::test_block_performance[Sync-vmlinux-5.10.238-True-libaio-bs4096-randwrite-1vcpu] defaultdict(<class 'list'>, {'fc_vcpu 0': [99.95329178526431, 99.95998553854952, 100.91893865958598, 99.95372053973948, 99.95989024751518, 99.96019994404158, 99.96074787336654, 99.9602237669306, 99.96100992864535, 99.95634078590423, 99.96043817344481, 99.96077169651657, 99.95996171577396, 99.96024758983143, 99.9610575752075, 99.959890247515, 99.96036670450454, 99.9598902475152, 99.96053346552394, 99.96077169651662, 99.96105757520755, 99.95905645871258, 99.96046199644755, 99.960152298297, 97.96083252045239, 100.95780491065754, 99.96086698923239, 99.95917557054712, 99.96003318413437, 99.96017612116383], 'firecracker': [75.96450175680081, 75.96958900929774, 74.93980593533604, 74.96529040480463, 74.96991768563629, 75.96975195747163, 75.9701683837586, 74.9701678251981, 76.96997764505684, 75.96681899728692, 74.97032863008363, 75.97018648935281, 74.96997128683046, 75.96978816827192, 75.9704037571575, 75.96951658811142, 74.97027502837844, 74.96991768563629, 75.97000543379829, 75.97018648935264, 75.97040375715775, 74.96929234403441, 75.96995111730011, 75.9697157467057, 74.97002488810134, 75.96824923970296, 75.9702589118162, 74.96938167791045, 75.9696252199425, 74.97013209087298], 'fc_api': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]})

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.
  • This functionality cannot be added in rust-vmm.

@Manciukic Manciukic marked this pull request as ready for review August 14, 2025 16:34
@codecov Codecov
Copy link

codecov bot commented Aug 14, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.35%. Comparing base (d974044) to head (6f3f117).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #5385      +/-   ##
==========================================
+ Coverage   82.31%   82.35%   +0.04%     
==========================================
  Files         265      265              
  Lines       30645    30645              
==========================================
+ Hits        25224    25238      +14     
+ Misses       5421     5407      -14
Flag Coverage Δ
5.10-c5n.metal 82.32% <ø> (ø)
5.10-m5n.metal 82.32% <ø> (+<0.01%) ⬆️
5.10-m6a.metal 81.60% <ø> (-0.01%) ⬇️
5.10-m6g.metal 78.92% <ø> (ø)
5.10-m6i.metal 82.32% <ø> (ø)
5.10-m7a.metal-48xl 81.59% <ø> (?)
5.10-m7g.metal 78.92% <ø> (ø)
5.10-m7i.metal-24xl 82.29% <ø> (?)
5.10-m7i.metal-48xl 82.30% <ø> (?)
5.10-m8g.metal-24xl 78.92% <ø> (?)
5.10-m8g.metal-48xl 78.92% <ø> (?)
6.1-c5n.metal 82.36% <ø> (ø)
6.1-m5n.metal 82.36% <ø> (ø)
6.1-m6a.metal 81.64% <ø> (-0.01%) ⬇️
6.1-m6g.metal 78.92% <ø> (ø)
6.1-m6i.metal 82.36% <ø> (ø)
6.1-m7a.metal-48xl 81.64% <ø> (?)
6.1-m7g.metal 78.92% <ø> (ø)
6.1-m7i.metal-24xl 82.38% <ø> (?)
6.1-m7i.metal-48xl 82.38% <ø> (?)
6.1-m8g.metal-24xl 78.92% <ø> (?)
6.1-m8g.metal-48xl 78.92% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Manciukic
Copy link
Contributor Author

A/B run to verify this doesn't happen again on A/B: https://buildkite.com/firecracker/performance-a-b-tests/builds/678/steps/canvas

@Manciukic Manciukic added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Aug 14, 2025
kalyazin
kalyazin reviewed Aug 15, 2025
View reviewed changes
roypat
roypat reviewed Aug 15, 2025
View reviewed changes
@Manciukic Manciukic force-pushed the fix-cpu-monitor-v2 branch 2 times, most recently from 000a8f5 to 5eef763 Compare August 15, 2025 11:59
@Manciukic
fix(tests/cpu-monitor): use psutil.Process for threads to fix errors
6f3f117 
The previous fix didn't work because the read can also return
ProcessLookupError. There is also the issue of pid recycling.

For this reason, use a single psutil.Process to verify the process still
exists, and catch any exception just in case.

Signed-off-by: Riccardo Mancini <[email protected]>
@Manciukic
Copy link
Contributor Author

Pushed new revision. New A/B to test the change: https://buildkite.com/firecracker/performance-a-b-tests/builds/679/steps/canvas

@Manciukic Manciukic merged commit 8e72870 into firecracker-microvm:main Aug 15, 2025
8 checks passed
@Manciukic Manciukic deleted the fix-cpu-monitor-v2 branch August 15, 2025 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kalyazin kalyazin kalyazin approved these changes

+1 more reviewer

@roypat roypat roypat approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Manciukic @kalyazin @roypat